Skip to content

Conversation

@adoroszlai
Copy link
Contributor

@adoroszlai adoroszlai commented Dec 20, 2019

What changes were proposed in this pull request?

ozonefs and tools modules in hadoop-ozone have a mix of unit and integration tests. This PR proposes to

  1. switch dependency order: let integration-test depend on these modules instead of the other way around
  2. move integration tests (those that use Mini*Cluster) from these modules to integration-test
  3. let unit check run remaining tests in these modules

This improves code coverage in CI.

https://issues.apache.org/jira/browse/HDDS-2785

How was this patch tested?

Clean CI run in fork: https://github.com/adoroszlai/hadoop-ozone/runs/358275984

Ran integration tests locally (with failures).

@adoroszlai adoroszlai self-assigned this Dec 20, 2019
@elek
Copy link
Member

elek commented Jan 6, 2020

Thanks the patch @adoroszlai. I like the approach to reverse the dependency (and simplify acceptance.sh and unit.sh), but I would prefer to keep the feature to separate certain type of integration tests as in this branch/runs:

https://github.com/apache/hadoop-ozone/runs/373475163

The test execution of integration-test project is already very slow, we can make it faster with separating contract/tools tests. I am not sure how to do it in this new structure. We have multiple options:

  1. put everything to the integration-test and use profiles to separate tests
  2. create separated integration-test/contract and integration-test/tools projects
  3. Keep the tests in the original projects and use the maven integration-test phase to execute these tests.

What is your preference?

@adoroszlai
Copy link
Contributor Author

Thanks @elek for bringing this up.

  1. put everything to the integration-test and use profiles to separate tests
  2. create separated integration-test/contract and integration-test/tools projects
  3. Keep the tests in the original projects and use the maven integration-test phase to execute these tests.

What is your preference?

I think option 1 would give us the most fine-grained control for grouping tests to ensure similar run time of the parallel jobs.

@elek
Copy link
Member

elek commented Jan 6, 2020

I think option 1 would give us the most fine-grained control for grouping tests to ensure similar run time of the parallel jobs.

In this case let me commit this one, and we can introduce the new profiles in the patch where the integration tests are enabled...

@elek elek closed this in 28cefc6 Jan 6, 2020
@adoroszlai adoroszlai deleted the HDDS-2785 branch January 6, 2020 15:10
@adoroszlai
Copy link
Contributor Author

Thanks @elek for reviewing and committing this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants